This invention relates to forming images with light, where the image is determined computationally from time-resolved measurements. The spatial resolution of the computed image is finer than the spatial support of the illuminations and finer than the spatial support of the sensitivities of the light meters. In an embodiment representing a limiting case, images are formed with a single illumination extending across the entire scene and each light meter sensitive to the entire scene.
Imaging is distinguished from other forms of sensing by having the goal of producing an image—a representation in one-to-one spatial correspondence with an object or scene. An image could appear on a continuous substrate such as film or paper and thus require uncountably-many numbers to be described. This invention relates instead to digital images—images described by finitely-many entries, where each entry is finitely-many numbers in discrete or continuous sets. The spatial correspondence of a digital image relates an entry in a digital image to a point, line, patch, or volume in the scene. Lines, patches, or volumes may overlap. Each entry in a digital image can represent brightness, reflectance, or other radiometric properties of a scene known to those of skill in the art. An entry in a digital image comprising a plurality of values may represent radiometric properties distinguished by wavelength or other distinctions known to those of skill in the art.
Traditional imaging involves illumination of the scene with a light source whose intensity does not vary appreciably with time. For example, the reflectance pattern on scene surfaces is acquired by using a lens to focus the incident light onto a plurality of light meters. Those of skill in the art will recognize that light meter can refer to any method of measuring light intensity, including but not limited to chemical and electronic methods, such as a PIN photodiode. Each light meter is sensitive to light originating at some subset of the scene, possibly with some variation of sensitivity with spatial location. The light meters are often in a regular two-dimensional (2D) array, but those skilled in the art will recognize that other arrangements of light meters are possible. Each light meter measures light intensity without time resolution. This means that while each light meter may have a sensitivity to incident light that varies with time—for example, using a shutter or any other method known to those skilled in the art—the light meter produces a single measurement, so variation of the incident light intensity with time cannot be inferred. Each entry of the digital image corresponds to one light meter. The number of light meters equals the number of pixels of the digital image; the number of pixels is also called the size of the digital image.
Many methods of imaging that use light sources and light meters in non-traditional ways are known to those skilled in the art. For example, dual photography [5] uses directed illumination and a single light meter. By Helmholtz reciprocity, directing light at one scene point, line, patch, or volume and measuring at one light meter is approximately equivalent to illuminating the entire scene and focusing reflected light only from the one scene point, line, patch, or volume on a light meter. Thus, by directing the illumination at a plurality of scene points, lines, patches, or volumes, one at a time, the effect of using a plurality of light meters is simulated. The number of distinct illumination points, lines, patches, or volumes equals the size of the digital image. Compressive dual photography [10], structured illumination [11], patterned light collection [12], and computational ghost imaging [13] methods are variations on traditional or dual photography that use simultaneous illumination of and measurement of incident light from a plurality of scene points, lines, patches, or volumes. Digital images are formed through computational means with spatial resolution that is not finer than the smaller of the spatial support of the illumination and the spatial support of the sensitivity of the light meters. These methods do not use time-resolved measurement by the light meters.
The use of time-resolved measurement by light meters, especially very high-speed sensing in photography, is associated with effectively stopping motion [2]. An alternative is to exploit the finite speed of light to associate travel time of light with distances. This is achieved in time-of-flight range measurement systems [3,4], which map time shifts in intensity-modulated light signals to distances. These systems and a generalization that employs multiply-reflected light [6] do not infer properties other than distances, such as brightness or reflectance.
A method is known in which a digital image of a surface not in the line of sight of the illumination source or light meters is produced with light reflected from an approximately Lambertian reflector surface that is in the line of sight of the illumination source, the surface to be imaged, and a set of light meters [1]. The reflector surface is illuminated with very short impulses of light using a 50-femtosecond Ti:Sapphire laser operating at 75 MHz and with a wavelength of 798 nm. Light that scatters from the reflector to the surface to be imaged, back to the reflector and then to a set of light meters that are optically focused on the reflector are measured with very fine time resolution of 20 picoseconds using a streak camera. Measurements that are obtained with six illumination target points chosen on the reflector are computationally processed to obtain 50-by-50 pixel digital images. This prior art is limited by the use of very short pulses of light (approximately 50 femtoseconds) since fast variation of illumination intensity is more difficult to achieve than slower variation. Furthermore, it is limited by the use of indirect illumination and assumptions of omnidirectional reflections from the reflector surface.